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Abstract — We consider a real-valued additive channel with an 
individual unknown noise sequence. We present a simple sequen- 
tial communication scheme based on the celebrated Schalkwijk- 
Kailath scheme, which varies the transmit power according to 
the power of the sequence, so that asymptotically the relation 
between the SNR and the rate matches the Gaussian channel 
capacity R « | log(l + SNR) for almost every noise sequence. 

I. Introduction 

We address the problem of communicating with feedback 
over the channel 

Vn -En ~t~ Sn ( 1 ) 

where the noise sequence s n can be any individual unknown 
sequence with arbitrary power. We show that the capacity 
of the equivalent Gaussian additive channel i log(l + P/N) 



with noise power N 



n 



s n and transmit power 



P = — x 2 n can be attained for asymptotically every 
noise sequence, without prior knowledge of its power. Note 
that when N is known in advance this channel becomes an 
additive arbitrarily varying channel (AVC) as treated in JTJ . 
This problem is in a way the real valued equivalent of the 
problem considered in 12, of the binary channel y n = x n ®e n 
with individual sequence e„ , where the error sequence e„ 
can be any unknown sequence. Using perfect feedback and 
common randomness, the authors show communication is 
possible at a rate approaching the capacity of the binary 
symmetric channel (BSC) with the same error probability 
1 — hb(e) where e equals the empirical error probability of 
the sequence (the relative number of T-s in e„) and hb(p) is 
the binary entropy function. The scheme achieving this rate is 
based on Horstein's feedback communication scheme for the 
BSC |3l] ■ These results were later extended to modulu additive 
and general discrete memoryless channels by Shayevitz B and 
Eswaran HE]. 

The scheme presented here is very similar to Schalkwijk- 
Kailath's 0. We use perfect feedback and common random- 
ness. Unlike J2) we exploit a property of the continuous 
channel and adapt the power of transmission rather than the 
rate. This is done mainly because adapting the power proves 
much simpler than adapting the rate: in the Schalkwijk-Kailath 
scheme the rate is expressed by the expansion factor of the 
error in sequential transmission, and changing the rate requires 
adapting this factor (e.g. according to an estimated channel 
behavior), while for the power adaptive scheme no estimation 
is required and the scheme is remarkably simple. On the other 



hand, adapting the power is not necessarily less practical than 
adapting the rate - for example in the uplink of cellular systems 
the power is adapted by a power control loop and the rate may 
be fixed e.g. in voice calls. This said, we do not intend to claim 
that the scheme is practical, since it strongly depends on the 
use of perfect feedback. 

We apply randomization only to the signs of the transmitted 
symbols (i.e. the total randomization required is n bits of infor- 
mation). Without randomization, this channel is symmetrizable 
for N > P (SNR < OdB), and therefore its AVC capacity is 

(see mOD). The scheme presented here can be modified 
to exclude randomization but in this case achieves only rate 

1 \og(P/N), which is less than the AVC capacity (T), so we 
do not discuss this case further. 

II. Statement of the main result 

The following theorem defines formally the properties of 
the proposed scheme: 

Theorem 1. Let Y = X + S, denote an additive channel 
where X, S and Y are n-length real random sequences 
denoting the input, an unknown predetermined interfering 
sequence and the output, respectively. Define P = i||X|| 2 
and N = -\\S\\ 2 . The state sequence may be determined by 
any fixed or randomized policy (possibly adversarial) which 
depends only on the past channel inputs and outputs, i.e. 
satisfying the Markov relation Vi : Si <-> ) «-> Xi 

(a fixed sequence is a particular case). 

For any e > 0, 8 > 0, N* > and R > 0, there is n 
large enough and a transmission scheme having fixed rate R 
and variable transmit power P, utilizing common randomness 
and perfect feedback, such that for any policy determining the 
state sequence with mean power E[N] < N* any of exp(ni?) 
messages can be sent with error probability less than e, and 
the average transmit power satisfies: 

E[P] < E[N] (exp (2R) - 1) + S (2) 



Or equivalently 



R>^og 



i 



E[P] - 5 



(3) 



where the expectations and the error probability are taken over 
common randomness and possibly the randomness of S. 

In other words, the scheme asymptotically achieves the 
Gaussian channel capacity for all state sequences within a ball 
of arbitrarily large size, by varying the mean transmit power. 



III. Presentation and analysis of the scheme 

A. The feedback transmission scheme 

As in the Schalkwijk-Kailath scheme we express a message 
m G {1, . . . , exp(ni?)} by a number in the unit interval 
[0, 1). We divide this interval into exp(ni?) disjoint message 
intervals of equal length and send the center of the interval 
9 = (m — i) • cxp(— nR) £ [0, 1). The receiver successively 
refines at each channel use an estimate of 9 denoted 9i and 
initialized to 0$ = |. The final estimate of 9 following 
the transmission is 9 n . Due to the perfect feedback the 
transmitter knows 6i. We define Di as a sequence of i.i.d. 
random variables uniformly distributed over { — 1, +1} known 
to the transmitter and receiver which comprises the common 
randomness. At each channel use i € {1, . . . , n} the transmit- 
ter sends: 



*A(0i-i-0) 



Di 



(4) 



Where e, = 6% — 9, and the receiver produces the estimate: 

h = k-i - Pot Di ■ Y, (5) 

Finally the decoded message is rh = round (9 n ■ exp(ni?)). 
The parameters a, (3 are fixed (depend only on the fixed rate 
and n) and will be specified later. 

In the analysis we treat 9 as a constant, and Xi,Yi, Di, Si 
as random variables. The randomness stems from Di and from 
possible randomness in Si, and all expectations in the sequel 
are taken with respect to this randomness. In the first part we 
examine the relation between the power of S and the power of 
X and find a bound on the transmit power and then develop a 
bound on the error probability. Finally we select a, f3 so that 
the conditions of the theorem are met. 

B. Power analysis 

We define P t = E[X 2 } and N = E[Sf], so that E[P] = 

By subtracting 8 from the two sides of eq.© and substi- 
tuting eq.© and the channel rule Y^ = X.- L + Si we have the 
following recursive relation: 

= ei _! - PatDtYi = ei_i - PaWiicr'Diei-! + S z ) = 
= (1 - /3)ej_i - PcfDiSt (6) 

and since Di is zero mean and independent of the past, ej_i 
and DiSi are uncorrelated, hence 



E[e*} = (l-P) 2 .E[el 1 }+p 2 a 2i E[S*} 



(7) 



We have P 1 = a~ 2 E[el] 
recursion: 



and the following 



P i+1 = E[X 2 +1 ) = a- 2 ^E[e 2 ] = 

= a- 2 (' i+1 > ((1 - P) 2 • £[ e 2_i] + P 2 a 2i E[S 2 ]) = 
= a- 2 (l - P) 2 ■ a- 2l £[eti] + a- 2 p 2 N l = 

= a- 2 (l~P) 2 -P t +a- 2 p 2 N t (8) 



Eq.® defines a linear relation between the sequences {Pi} 
and {Ni}. Pi can be regarded as the output of an infinite 
impulse response (IIR) filter operating on the input sequence 
Ni. Below we bound the transmit power for a given noise 
power by using the fact the impulse response of this filter is 
positive, and thus the contribution of each N to the integrated 
filter output Y17=i Pi i s bounded by the integrated impulse 
response, with equality for n — ► oo. This way the contribution 
of the sequence S can be bounded using only its total power 
(a bound which becomes tight for n — > oo unless the energy 
of S is concentrated at the end of the block). Solving eq.® 
we have 



Pi = Y,hi- j -N j + 1 i - l P 1 



(9) 



with 7 = a 2 (1 — P) 2 and hj = a 2 p 2 ■ 7 J 1 and therefore 



j 

n i—1 



e[p] = E P ' = EE h ^ ■ N i + E 

i— 1 j—1 i—1 



n n 



E E >>■ r \ • E-' i' 

j=l i=j + l i=l 

= E(^ E hi.] +±^a- 2 (6 



3=1 



i=j+l 



n / oo \ 1 oo 

<E UEM+jE^« 

3=1 \ i=l / i=l 



n ■ E[N] 



a~ 2 p 2 



1 of 



(10) 



l-a- 2 (l-/3) 2 4 1-7 

Minimizing the coefficient multiplying E[N] in the RHS with 
respect to P we obtain P = 1 — a 2 , and therefore 7 = a 2 and 

1 1 



E[P] < E[N] ■ {aT 2 - 1) 



(11) 



n 4a 2 (l-a 2 ) 

We now turn our attention to the error probability and we will 
return to this equation later. 

C. Error analysis 

A decoding error occurs if \6 n — 6\ = \e n \ > | ■ exp(— nR). 
Returning to eq.© we note that it defines a linear relation 
between the sequence {Ni} and {i?[e 2 ]}. Solving the equation 
and substituting p = 1 — a 2 we have the following bound on 
the mean square estimation error: 

n 

E[e 2 n ] = (1 - P) 2n ■ E[e 2 } + £(1 - pf^p 2 ^^ = 



1 " 

-f +p 2 Y J a 4n - 23 N. ] < 



3=1 



1 - 1 

< -a 4 " + P 2 a 2n N o < / 4 " + P 2 a 2n nN* (12) 



where the two inequalities are met with equality when the 
noise sequence has maximum energy which is concentrated on 



the last symbol (Sf = Si- n N*). Therefore from Chebychev 
inequality we have: 

P e = Pr(m ^ m) = Pr ||e n | > i • cxp(-?ii?)| < 

< < (a 4 ™ + A(3 2 a 2n nN*) exp(2ni?) < 

i • exp(-2ni?) 

< (1 + A(3 2 nN*) ■ (a ■ exp(i?)) 2n (13) 

D. Setting the parameters 

From section IIII-BI we have that the optimal value of f3 is 
j3 = 1 — a 2 . From eq.([T3l we have that P e — >0 for any 
fixed a chosen as a < exp(— R). Therefore there exists 
a sequence a n — > exp(— i?) such that for n large enough 
P e < e. Substituting this sequence into eq.dTTb and using the 
continuity of the bound as function of a we have 

E[P) <^ E[N) ■ (a~ 2 - 1)+ 

+ V^W^)— N - {cMm) ~ l) (14) 

where the convergence is uniform in E[N] since E[N] < N* 
is bounded. Therefore for any 6 there is n large enough such 
that E[P] < E[N] • (exp(-2iJ) - 1) + 5, which completes the 
proof of Theorem [TJ 

IV. Discussion 

The scheme we presented is almost equivalent to the 
Schalkwijk-Kailath scheme Q, except we did not assume 
the noise is Gaussian, and we added scrambling (Di). The 
parameter j3 is equivalent to the shrinkage parameter created 
by the maximum likelihood estimator in the original scheme, 
and a is the same parameter referred to as a -1 in the original 
scheme. The main difference is in the error and power analysis. 
Schalkwijk-Kailath's analysis assumes Gaussian noise and 
shows an error probability which decays doubly-exponentially 
in the block length n. In our analysis the bound on the error 
probability decays only exponentially (if a is kept constant) 
which comes from using the Chebychev inequality instead of 
the Gaussian error function. 

This scheme can be operated also over any random sequence 
which is independent of Xi (but may be dependent on the 
past X^ _1 ). If the sequence has zero mean conditioned on 
the past then the scrambling Di is not required, and our 
scheme is equivalent to Schalkwijk-Kailath's. Therefore we 
can conclude that the Schalkwijk-Kailath scheme achieves the 
Gaussian capacity | log(l+P/iV) for any (conditionally) zero 
mean noise process (not necessarily i.i.d), and in this case 
the average transmit power is constant (rather than arbitrary). 
If the noise process is further assumed to be ergodic, then 
since the impulse response generating the transmitted power 
is finite, as n — > oo eq.dTTb will approximately hold in equality 
with high probability. Thus taking the limit of eq. dTTb we 
have P w N(a~ 2 — 1) is nearly constant. In other words, 
if the process is ergodic then the Schalkwijk-Kailath scheme 
will approximately generate a fixed power. The result that 



the Schalkwijk-Kailath scheme achieves the Gaussian capacity 
for any memoryless i.i.d. Gaussian additive noise channel by 
varying the power has already been shown in |9] and extended 
to general i.i.d. noise in ifTOl using different tools. In both cases 
considered in these publications only a decrease, or a bounded 
increase in the noise variance is possible, otherwise the error 
probability is compromised. 

The role of the scrambler Di is to prevent the sequence 
S from building up a large error or transmit power through 
coherent combining of errors between iterations. Since the 
recursive relation eq.© defines a frequency selective transfer 
function (pertaining to the fourier transform of the sequences), 
a possible interpretation of the scrambling is as a means to 
flatten the spectrum of S and thus prevent maximization of 
the error by matching the spectrum of S to this function. 

Two weaknesses in the scheme are the limitation of the 
noise sequences to mean power N*, and the fact the transmit 
power is attained on the average rather than per transmission. 
The first limitation stems from the error analysis. Since the 
estimation error is linearly dependent on the input, then for 
example by determining the last element of the state sequence, 
any desired error can be generated. Hence, errors cannot be 
avoided unless limitations on the sequence are imposed. This 
is true for all symbols, but the strongest effect is in the last 
symbols (see eq.([T2l. Intuitively, one can say that when strong 
interference is present toward the end of the transmission, 
the scheme fails to build the suitable transmit power in time 
to cope with it. On the other hand, the expansion factor a 
forces the noise sequence to use asymptotically infinite power 
in order to cause an error, and therefore we are able to let 
N* — > oo (in an exponential rate, see eq. (fT2l ) but are not able 
to dismiss it completely. To see that no scheme can satisfy 
Theorem [TJ if the noise power constraint is removed, consider 
an adversary who computes at each symbol the maximum 
power the transmitter can use in the next symbol while obeying 
eq.©, and then applies to Si a Gaussian noise with power 
K times larger. This guarantees the mutual information per 
symbol will be at most | log (l + K^ 1 ) which tends to zero 
as K — + oo, therefore no positive rate can be sent reliably. 

The second weakness is that the power is attained only on 
the average, rather than always or "almost always". If one 
considers for example a noise sequence in which only two 
symbols are non zero, then the transmit power will fluctuate 
between two values (representing positive and negative co- 
herent combining). The law of large numbers applies only if 
the interfering sequence is constant or ergodic over multiple 
transmissions, but does not apply if we allow the sequence 
to be determined arbitrarily. This issue may be resolved by a 
more elaborate scheme. 

In lfTTIlfT2l we use a fixed power, rate adaptive scheme based 
on random rateless coding and success/fail feedback, which 
asymptotically attains the Gaussian capacity when used with 
the additive channel discussed here. The design of sequential 
schemes that achieve this goal (without random coding) is of 
interest. Another interesting direction is to extend the scheme 
to non-additive channels or to completely unspecified models 



(as we considered in lfTT1lfT2l ). 



References 

[I] I. Csiszar and P. Narayan, "Capacity of the Gaussian arbitrarily varying 
channel," Information Theory, IEEE Transactions on , vol.37, no.l, pp.18- 
26, Jan 1991 

[2] O. Shayevitz and M. Feder, "Communicating using Feedback over a 
Binary Channel with Arbitrary Noise Sequence", International Symposium 
on Information Theory (ISIT), Adelaide, Australia, September 2005. 

[3] M. Horstein, Sequential transmission using noiseless feedback, IEEE 
Trans. Info. Theory, pp. 136143, July 1963. 

[4] Ofer Shayevitz and Meir Feder, "Achieving the Empirical Capacity 
Using Feedback Part I: Memoryless Additive Models", Dept. of Electrical 
Engineering Systems, Tel Aviv University, Tel Aviv 69978, Israel 

[5] Krishnan Eswaran, Anand D. Sarwate, Anant Sahai, and Michael Gast- 
par, "Limited feedback achieves the empirical capacity," Department of 
Electrical Engineering and Computer Sciences, University of California, 
larXiv:071 1.0237V 1 [cs.IT] 2 Nov 2007. 

[6] K. Eswaran and A.D. Sarwate and A. Sahai and M. Gastpar, "Using zero- 
rate feedback on binary additive channels with individual noise sequences," 
Proceedings of the 2007 Infernational Symposium on Information Theory 
(ISIT 2007), Nice, France, June 2007 

[7] J. P. M. Schalkwijk, A coding scheme for additive noise channels with 
feedback part II: Band-limited siganls, IEEE Trans. Info. Theory, vol. IT- 
12, pp. 183 189, 1966. 

[8] I. Csiszar and P. Narayan, "The Capacity of the Arbitrarily Varying Chan- 
nel Revisited : Positivity, Constraints", IEEE Transactions On Information 
Theory, Vol. 34, No. 2, March 1988 

[9] Y.H. Kim, A. Lapidoth and T. Weissman, "Robustness and Sensitivity 
of the Schalkwijk-Kailath Scheme", Kailath Colloquium on Feedback 
Communication, Stanford, CA, July 2006. 

[10] Shayevitz, O.; Feder, M., "The posterior matching feedback scheme: 
Capacity achieving and error analysis," Information Theory, 2008. ISIT 
2008. IEEE International Symposium on , vol., no., pp. 900-904, 6-11 July 
2008 

[II] Yuval Lomnitz and Meir Feder, "Feedback communication over in- 
dividual channels," submitted to the 2009 International Symposium on 
Information Theory (ISIT), Seoul 

[12] Yuval Lomnitz a nd Meir F eder, "Communication over 
Individual Channels," [a rXiv:0901.1473yl [cs.IT], 11 Jan 2009, 
|http://arxiv.org/abs/0901 . 1473vl | 

[13] Shayevitz, Ofer; Feder, Meir, "Communication with Feedback via Pos- 
terior Matching," Information Theory, 2007. ISIT 2007. IEEE International 
Symposium on , vol., no., pp. 391-395, 24-29 June 2007 



